Introduction to Computer Vision: Plant Seedlings Classification¶
Problem Statement¶
Context¶
In recent times, the field of agriculture has been in urgent need of modernizing, since the amount of manual work people need to put in to check if plants are growing correctly is still highly extensive. Despite several advances in agricultural technology, people working in the agricultural industry still need to have the ability to sort and recognize different plants and weeds, which takes a lot of time and effort in the long term. The potential is ripe for this trillion-dollar industry to be greatly impacted by technological innovations that cut down on the requirement for manual labor, and this is where Artificial Intelligence can actually benefit the workers in this field, as the time and energy required to identify plant seedlings will be greatly shortened by the use of AI and Deep Learning. The ability to do so far more efficiently and even more effectively than experienced manual labor, could lead to better crop yields, the freeing up of human inolvement for higher-order agricultural decision making, and in the long term will result in more sustainable environmental practices in agriculture as well.
Objective¶
The aim of this project is to Build a Convolutional Neural Netowrk to classify plant seedlings into their respective categories.
Data Dictionary¶
The Aarhus University Signal Processing group, in collaboration with the University of Southern Denmark, has recently released a dataset containing images of unique plants belonging to 12 different species.
The dataset can be download from Olympus.
The data file names are:
- images.npy
- Labels.csv
Due to the large volume of data, the images were converted to the images.npy file and the labels are also put into Labels.csv, so that you can work on the data/project seamlessly without having to worry about the high data volume.
The goal of the project is to create a classifier capable of determining a plant's species from an image.
List of Species
- Black-grass
- Charlock
- Cleavers
- Common Chickweed
- Common Wheat
- Fat Hen
- Loose Silky-bent
- Maize
- Scentless Mayweed
- Shepherds Purse
- Small-flowered Cranesbill
- Sugar beet
Note: Please use GPU runtime on Google Colab to execute the code faster.¶
Importing necessary libraries¶
# Installing the libraries with the specified version.
# uncomment and run the following line if Google Colab is being used
#!pip install tensorflow==2.15.0 scikit-learn==1.2.2 seaborn==0.13.1 matplotlib==3.7.1 numpy==1.25.2 pandas==1.5.3 opencv-python==4.8.0.76 -q --user
# Installing the libraries with the specified version.
# uncomment and run the following lines if Jupyter Notebook is being used
#!pip install tensorflow==2.13.0 scikit-learn==1.2.2 seaborn==0.11.1 matplotlib==3.3.4 numpy==1.24.3 pandas==1.5.2 opencv-python==4.8.0.76 -q --user
Note: After running the above cell, kindly restart the notebook kernel and run all cells sequentially from the start again.
import os
import numpy as np # Importing numpy for Matrix Operations
import pandas as pd # Importing pandas to read CSV files
import matplotlib.pyplot as plt # Importting matplotlib for Plotting and visualizing images
import math # Importing math module to perform mathematical operations
import cv2 # Importing openCV for image processing
import seaborn as sns # Importing seaborn to plot graphs
# Tensorflow modules
import tensorflow as tf
from tensorflow.keras.preprocessing.image import ImageDataGenerator # Importing the ImageDataGenerator for data augmentation
from tensorflow.keras.models import Sequential # Importing the sequential module to define a sequential model
from tensorflow.keras.layers import Dense,Dropout,Flatten,Conv2D,MaxPooling2D,BatchNormalization # Defining all the layers to build our CNN Model
from tensorflow.keras.optimizers import Adam,SGD # Importing the optimizers which can be used in our model
from sklearn import preprocessing # Importing the preprocessing module to preprocess the data
from sklearn.model_selection import train_test_split # Importing train_test_split function to split the data into train and test
from sklearn.metrics import confusion_matrix # Importing confusion_matrix to plot the confusion matrix
from sklearn.preprocessing import LabelBinarizer
# Display images using OpenCV
import cv2 # Importing cv2
from sklearn.model_selection import train_test_split
from tensorflow.keras import backend
from keras.callbacks import ReduceLROnPlateau
from keras import layers
import random
# Ignore warnings
import warnings
warnings.filterwarnings('ignore')
Loading the dataset¶
# Load image file of the dataset
images = np.load("D:/AIML UT AUSTIN/Introduction to Computer Vision/Project5/images.npy")
# Load the labels file of the dataset
labels = pd.read_csv("D:/AIML UT AUSTIN/Introduction to Computer Vision/Project5/Labels.csv")
Data Overview¶
Understand the shape of the dataset¶
images.shape
(4750, 128, 128, 3)
- Total 4750 images are there of Shape 128*128 and 3 channel
labels.shape
(4750, 1)
# Create copy of data
images_bgr = images.copy()
Exploratory Data Analysis¶
- EDA is an important part of any project involving data.
- It is important to investigate and understand the data better before building a model with it.
- A few questions have been mentioned below which will help you understand the data better.
- A thorough analysis of the data, in addition to the questions mentioned below, should be done.
- How are these different category plant images different from each other?
- Is the dataset provided an imbalance? (Check with using bar plots)
def plot_images(images, labels):
num_classes = 12
categories = np.unique(labels)
keys = dict(labels['Label'])
rows = 5
cols = 5
fig = plt.figure(figsize=(12,16))
for i in range(cols):
for j in range(rows):
random_index = np.random.randint(0, len(images))
ax = fig.add_subplot(rows, cols, i * rows + j + 1)
ax.imshow(images[random_index, :])
ax.set_title(keys[random_index])
plt.show()
plot_images(images_bgr, labels)
Checking data imbalance¶
order = labels['Label'].value_counts().index
sns.countplot(labels['Label'], order=order)
plt.xticks(rotation=90)
plt.show()
- As we can see data has variations in counts of plant species. But we have sufficients counts to train the model
Data Pre-Processing¶
Convert the BGR images to RGB images¶
for i in range(len(images)):
images[i] = cv2.cvtColor(images[i], cv2.COLOR_BGR2RGB)
# Plot images after BGR to RGB for better understanding
plot_images(images, labels)
Resize the images¶
# Resize images to 128*128 so as to keep the quality of images and make sure have same size images in data
images_resized=[]
height = 128 # Complete the code to define the height as 128
width = 128 # Complete the code to define the width as 128
dimensions = (width, height)
for i in range(len(images)):
images_resized.append( cv2.resize(images[i], dimensions, interpolation=cv2.INTER_LINEAR))
plt.imshow(images_resized[30])
<matplotlib.image.AxesImage at 0x1b4544f63d0>
images.shape
(4750, 128, 128, 3)
Apply image processing on the images¶
- Cconvert image from RGB to HSV
Applymasking for images
- Apply Segmentation
- Apply Sharpeninging
def create_mask_for_plant(image):
image_hsv = cv2.cvtColor(image, cv2.COLOR_RGB2HSV)
sensitivity = 35
lower_hsv = np.array([60 - sensitivity, 100, 50])
upper_hsv = np.array([60 + sensitivity, 255, 255])
mask = cv2.inRange(image_hsv, lower_hsv, upper_hsv)
kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (11,11))
mask = cv2.morphologyEx(mask, cv2.MORPH_CLOSE, kernel)
return mask
def segment_plant(image):
mask = create_mask_for_plant(image)
output = cv2.bitwise_and(image, image, mask = mask)
return output
def sharpen_image(image):
image_blurred = cv2.GaussianBlur(image, (0, 0), 3)
image_sharp = cv2.addWeighted(image, 1.5, image_blurred, -0.5, 0)
return image_sharp
# put all those functions together.
def segment(img):
#image_mask = create_mask_for_plant(img)
image_segmented = segment_plant(img)
image_sharpen = sharpen_image(image_segmented)
return image_sharpen
Itrate on all images¶
# and `images_resized` is a list of resized images
# Initialize an empty list to store processed images
processed_data_color = []
# Iterate through the images
for indx, image in enumerate(images_resized):
try:
# Process the image using the segment function
processed_image = segment(image)
# Append the processed image to the array
processed_data_color.append(processed_image)
except Exception as e:
# Print the index and error details for debugging
print(f"Error processing image at index {indx}: {e}")
continue
# Convert the list to a NumPy array for further processing if needed
import numpy as np
processed_data_color = np.array(processed_data_color)
processed_data_color.shape
(4750, 128, 128, 3)
plt.imshow(images[2000])
<matplotlib.image.AxesImage at 0x1b4545563d0>
plt.imshow(processed_data_color[2000])
<matplotlib.image.AxesImage at 0x1b42f478550>
Data Preparation for Modeling¶
- Before you proceed to build a model, you need to split the data into train, test, and validation to be able to evaluate the model that you build on the train data
- You'll have to encode categorical features and scale the pixel values.
- You will build a model using the train data and then check its performance
Split the dataset¶
X_temp, X_test, y_temp, y_test = train_test_split(processed_data_color, labels, test_size=0.1, random_state=42, stratify=labels)
X_train, X_val, y_train, y_val = train_test_split(X_temp, y_temp, test_size=0.1, random_state=42, stratify=y_temp)
print(X_train.shape,y_train.shape)
print(X_val.shape,y_val.shape)
print(X_test.shape,y_test.shape)
(3847, 128, 128, 3) (3847, 1) (428, 128, 128, 3) (428, 1) (475, 128, 128, 3) (475, 1)
Encode the target labels¶
# Convert labels from names to one hot vectors.
lb = LabelBinarizer()
y_train_encoded = lb.fit_transform(y_train)
y_val_encoded=lb.transform(y_val)
y_test_encoded=lb.transform(y_test)
y_train_encoded.shape
(3847, 12)
y_val_encoded.shape
(428, 12)
y_test_encoded.shape
(475, 12)
Data Normalization¶
# Normalizing the image pixels
X_train_normalized = X_train.astype('float32')/255.0
X_val_normalized = X_val.astype('float32')/255.0
X_test_normalized = X_test.astype('float32')/255.0
Model Building¶
Model 0 - Base Model with Convolution and Pulling layers¶
# Clearing Backend
from tensorflow.keras import backend
backend.clear_session()
# Fixing the seed for random number generators
import random
np.random.seed(42)
random.seed(42)
tf.random.set_seed(42)
# Initializing a sequential model
model = Sequential()
# Adding first conv layer with 128 filters and Kernel size 3*3, padding 'same'
model.add(Conv2D(32, kernel_size=(3, 3), activation='relu', input_shape=(128,128,3)))
# Adding max pooling to reduce the size of output of first conv layer
model.add(MaxPooling2D((2, 2)))
# Add two convolution and max-pooling layers activation = relu
model.add(Conv2D(64, kernel_size=(3, 3), activation='relu'))
model.add(MaxPooling2D((2, 2)))
model.add(Conv2D(64, kernel_size=(3, 3), activation='relu'))
model.add(MaxPooling2D((2, 2)))
model.add(Conv2D(64, kernel_size=(3, 3), activation='relu'))
model.add(MaxPooling2D((2, 2)))
# flattening the output of the conv layer after max pooling to make it ready for creating dense connections
model.add(Flatten())
# Adding a fully connected dense layer with 16 neurons
model.add(Dense(64, activation='relu'))
model.add(Dropout(0.3))
# Adding the output layer with 12 neurons and activation functions as softmax since this is a multi-class classification problem
model.add(Dense(12, activation='softmax'))
# Using SGD Optimizer
# opt = SGD(learning_rate=0.01, momentum=0.9)
opt=Adam()
# Compile model
model.compile(optimizer=opt, loss='categorical_crossentropy', metrics=['accuracy'])
# Generating the summary of the model
model.summary()
Model: "sequential"
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┓ ┃ Layer (type) ┃ Output Shape ┃ Param # ┃ ┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━┩ │ conv2d (Conv2D) │ (None, 126, 126, 32) │ 896 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ max_pooling2d (MaxPooling2D) │ (None, 63, 63, 32) │ 0 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ conv2d_1 (Conv2D) │ (None, 61, 61, 64) │ 18,496 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ max_pooling2d_1 (MaxPooling2D) │ (None, 30, 30, 64) │ 0 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ conv2d_2 (Conv2D) │ (None, 28, 28, 64) │ 36,928 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ max_pooling2d_2 (MaxPooling2D) │ (None, 14, 14, 64) │ 0 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ conv2d_3 (Conv2D) │ (None, 12, 12, 64) │ 36,928 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ max_pooling2d_3 (MaxPooling2D) │ (None, 6, 6, 64) │ 0 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ flatten (Flatten) │ (None, 2304) │ 0 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ dense (Dense) │ (None, 64) │ 147,520 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ dropout (Dropout) │ (None, 64) │ 0 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ dense_1 (Dense) │ (None, 12) │ 780 │ └──────────────────────────────────────┴─────────────────────────────┴─────────────────┘
Total params: 241,548 (943.55 KB)
Trainable params: 241,548 (943.55 KB)
Non-trainable params: 0 (0.00 B)
Fitting the model on training data¶
history = model.fit(
train_datagen1.flow(X_train_normalized, y_train_encoded, seed=42, shuffle=True),
epochs=40,
validation_data=(X_val_normalized, y_val_encoded),
batch_size=32,
verbose=2)
Epoch 1/40 121/121 - 13s - 106ms/step - accuracy: 0.2880 - loss: 2.1164 - val_accuracy: 0.4252 - val_loss: 1.6812 Epoch 2/40 121/121 - 11s - 94ms/step - accuracy: 0.4268 - loss: 1.6503 - val_accuracy: 0.4836 - val_loss: 1.4830 Epoch 3/40 121/121 - 12s - 97ms/step - accuracy: 0.4931 - loss: 1.4285 - val_accuracy: 0.5304 - val_loss: 1.3171 Epoch 4/40 121/121 - 12s - 97ms/step - accuracy: 0.5529 - loss: 1.2550 - val_accuracy: 0.5701 - val_loss: 1.2039 Epoch 5/40 121/121 - 12s - 99ms/step - accuracy: 0.5851 - loss: 1.1478 - val_accuracy: 0.6192 - val_loss: 1.0820 Epoch 6/40 121/121 - 12s - 98ms/step - accuracy: 0.6296 - loss: 1.0271 - val_accuracy: 0.7126 - val_loss: 0.8760 Epoch 7/40 121/121 - 11s - 94ms/step - accuracy: 0.6652 - loss: 0.9314 - val_accuracy: 0.7407 - val_loss: 0.7630 Epoch 8/40 121/121 - 11s - 93ms/step - accuracy: 0.7091 - loss: 0.8156 - val_accuracy: 0.6939 - val_loss: 0.8779 Epoch 9/40 121/121 - 11s - 92ms/step - accuracy: 0.7408 - loss: 0.7504 - val_accuracy: 0.7593 - val_loss: 0.7003 Epoch 10/40 121/121 - 11s - 91ms/step - accuracy: 0.7663 - loss: 0.6693 - val_accuracy: 0.7991 - val_loss: 0.5737 Epoch 11/40 121/121 - 11s - 94ms/step - accuracy: 0.7835 - loss: 0.6137 - val_accuracy: 0.8201 - val_loss: 0.5372 Epoch 12/40 121/121 - 11s - 90ms/step - accuracy: 0.7936 - loss: 0.5965 - val_accuracy: 0.8318 - val_loss: 0.5336 Epoch 13/40 121/121 - 11s - 91ms/step - accuracy: 0.8079 - loss: 0.5629 - val_accuracy: 0.7944 - val_loss: 0.6155 Epoch 14/40 121/121 - 11s - 91ms/step - accuracy: 0.8139 - loss: 0.5184 - val_accuracy: 0.8435 - val_loss: 0.4824 Epoch 15/40 121/121 - 11s - 91ms/step - accuracy: 0.8154 - loss: 0.5156 - val_accuracy: 0.8645 - val_loss: 0.4784 Epoch 16/40 121/121 - 11s - 90ms/step - accuracy: 0.8180 - loss: 0.5092 - val_accuracy: 0.8598 - val_loss: 0.4528 Epoch 17/40 121/121 - 11s - 90ms/step - accuracy: 0.8352 - loss: 0.4700 - val_accuracy: 0.8435 - val_loss: 0.4927 Epoch 18/40 121/121 - 11s - 90ms/step - accuracy: 0.8326 - loss: 0.4749 - val_accuracy: 0.8575 - val_loss: 0.4768 Epoch 19/40 121/121 - 11s - 90ms/step - accuracy: 0.8417 - loss: 0.4497 - val_accuracy: 0.8481 - val_loss: 0.4723 Epoch 20/40 121/121 - 11s - 91ms/step - accuracy: 0.8500 - loss: 0.4228 - val_accuracy: 0.8341 - val_loss: 0.4691 Epoch 21/40 121/121 - 11s - 91ms/step - accuracy: 0.8529 - loss: 0.4349 - val_accuracy: 0.8598 - val_loss: 0.4427 Epoch 22/40 121/121 - 11s - 91ms/step - accuracy: 0.8477 - loss: 0.4274 - val_accuracy: 0.8341 - val_loss: 0.4583 Epoch 23/40 121/121 - 11s - 90ms/step - accuracy: 0.8570 - loss: 0.4013 - val_accuracy: 0.8715 - val_loss: 0.4059 Epoch 24/40 121/121 - 11s - 89ms/step - accuracy: 0.8687 - loss: 0.3739 - val_accuracy: 0.8762 - val_loss: 0.3943 Epoch 25/40 121/121 - 11s - 90ms/step - accuracy: 0.8602 - loss: 0.3775 - val_accuracy: 0.8575 - val_loss: 0.4226 Epoch 26/40 121/121 - 11s - 90ms/step - accuracy: 0.8591 - loss: 0.3857 - val_accuracy: 0.8668 - val_loss: 0.3998 Epoch 27/40 121/121 - 11s - 92ms/step - accuracy: 0.8695 - loss: 0.3724 - val_accuracy: 0.8668 - val_loss: 0.3896 Epoch 28/40 121/121 - 11s - 90ms/step - accuracy: 0.8726 - loss: 0.3514 - val_accuracy: 0.8692 - val_loss: 0.3669 Epoch 29/40 121/121 - 11s - 90ms/step - accuracy: 0.8734 - loss: 0.3422 - val_accuracy: 0.8575 - val_loss: 0.4220 Epoch 30/40 121/121 - 11s - 92ms/step - accuracy: 0.8674 - loss: 0.3653 - val_accuracy: 0.8668 - val_loss: 0.4130 Epoch 31/40 121/121 - 11s - 91ms/step - accuracy: 0.8789 - loss: 0.3444 - val_accuracy: 0.8692 - val_loss: 0.4274 Epoch 32/40 121/121 - 11s - 93ms/step - accuracy: 0.8807 - loss: 0.3299 - val_accuracy: 0.8762 - val_loss: 0.3678 Epoch 33/40 121/121 - 12s - 95ms/step - accuracy: 0.8838 - loss: 0.3126 - val_accuracy: 0.8738 - val_loss: 0.3993 Epoch 34/40 121/121 - 11s - 94ms/step - accuracy: 0.8822 - loss: 0.3108 - val_accuracy: 0.8621 - val_loss: 0.4016 Epoch 35/40 121/121 - 11s - 94ms/step - accuracy: 0.8885 - loss: 0.2952 - val_accuracy: 0.8668 - val_loss: 0.3738 Epoch 36/40 121/121 - 11s - 93ms/step - accuracy: 0.8835 - loss: 0.3049 - val_accuracy: 0.8715 - val_loss: 0.3782 Epoch 37/40 121/121 - 11s - 92ms/step - accuracy: 0.8926 - loss: 0.2863 - val_accuracy: 0.8808 - val_loss: 0.3587 Epoch 38/40 121/121 - 11s - 94ms/step - accuracy: 0.8913 - loss: 0.2821 - val_accuracy: 0.8668 - val_loss: 0.4355 Epoch 39/40 121/121 - 11s - 92ms/step - accuracy: 0.8804 - loss: 0.3137 - val_accuracy: 0.8832 - val_loss: 0.4095 Epoch 40/40 121/121 - 11s - 94ms/step - accuracy: 0.8958 - loss: 0.2745 - val_accuracy: 0.8902 - val_loss: 0.3714
Plot Accuracy graph¶
plt.plot(history.history['accuracy'])
plt.plot(history.history['val_accuracy'])
plt.title('Model Accuracy')
plt.ylabel('Accuracy')
plt.xlabel('Epoch')
plt.legend(['Train', 'Validation'], loc='upper left')
plt.show()
Evaluate the model on Test Data¶
test_accuracy0 = model.evaluate(X_test_normalized, y_test_encoded, verbose=1)
15/15 ━━━━━━━━━━━━━━━━━━━━ 0s 18ms/step - accuracy: 0.8846 - loss: 0.4190
- Model is giving around 89% accuracy on Training and Validation set and around 88% on test set
- Model is generalizing well on data
Generating Predictions on Test Data¶
y_pred = model.predict(X_test_normalized)
15/15 ━━━━━━━━━━━━━━━━━━━━ 0s 23ms/step
Plot Confusion Matrix¶
# Obtaining the categorical values from y_test_encoded and y_pred
y_pred_arg=np.argmax(y_pred,axis=1)
y_test_arg=np.argmax(y_test_encoded,axis=1)
# Plotting the Confusion Matrix using confusion matrix() function which is also predefined tensorflow module
confusion_matrix = tf.math.confusion_matrix(y_test_arg,y_pred_arg)
f, ax = plt.subplots(figsize=(10, 8))
sns.heatmap(
confusion_matrix,
annot=True,
linewidths=.4,
fmt="d",
square=True,
ax=ax
)
# Setting the labels to both the axes
ax.set_xlabel('Predicted labels');ax.set_ylabel('True labels');
ax.set_title('Confusion Matrix');
ax.xaxis.set_ticklabels(list(lb.classes_),rotation=90)
ax.yaxis.set_ticklabels(list(lb.classes_),rotation=0)
plt.show()
# Plotting Classification Report
from sklearn.metrics import classification_report
print(classification_report(y_test_arg, y_pred_arg, target_names=list(lb.classes_)))
precision recall f1-score support
Black-grass 0.37 0.27 0.31 26
Charlock 0.95 0.90 0.92 39
Cleavers 0.92 0.83 0.87 29
Common Chickweed 0.94 0.98 0.96 61
Common wheat 0.78 0.82 0.80 22
Fat Hen 0.96 0.94 0.95 48
Loose Silky-bent 0.75 0.89 0.82 65
Maize 0.88 1.00 0.94 22
Scentless Mayweed 0.94 0.94 0.94 52
Shepherds Purse 1.00 0.74 0.85 23
Small-flowered Cranesbill 0.96 0.96 0.96 50
Sugar beet 0.92 0.92 0.92 38
accuracy 0.88 475
macro avg 0.86 0.85 0.85 475
weighted avg 0.88 0.88 0.88 475
Comments¶
- Recall is very low for other classes recall is good
Precision is low for Black grass but all other classes have good preicision and f1 score* Even from confusion matrix, we see model did not perform well for black gra, it is misclassifying black grass as loose silky bent
- Let's see if we can improve performance furtherher
Model Performance Improvement¶
Model 1 - Model with Data Augmentation¶
Data Augmentation¶
Remember, data augmentation should not be used in the validation/test data set.
- As we can see, our initial model appears to be not fitting well. Therefore we'll try to address this problem with data augmentation to check if we can improve the model's performance.
# set the rotation_range to 20
train_datagen1 = ImageDataGenerator(rotation_range=20, # randomly rotate images in the range
horizontal_flip=True, # randomly flip images horizontally
vertical_flip=True # randomly flip images vertically
)
Let's clear the previous model history and set the seed for random number generators in Numpy, the Random library in Python, and in TensorFlow to be able to reproduce the same results every time we run the code.
# Clearing Backend
from tensorflow.keras import backend
backend.clear_session()
# Fixing the seed for random number generators
import random
np.random.seed(42)
random.seed(42)
tf.random.set_seed(42)
# Initializing a sequential model
model1 = Sequential()
# Adding first conv layer with 128 filters and Kernel size 3*3, padding 'same'
model1.add(Conv2D(32, kernel_size=(3, 3), activation='relu', input_shape=(128,128,3)))
# Adding max pooling to reduce the size of output of first conv layer
model1.add(MaxPooling2D((2, 2)))
# Add two convolution and max-pooling layers activation = relu
model1.add(Conv2D(64, kernel_size=(3, 3), activation='relu'))
model1.add(MaxPooling2D((2, 2)))
model1.add(Conv2D(64, kernel_size=(3, 3), activation='relu'))
model1.add(MaxPooling2D((2, 2)))
model1.add(Conv2D(64, kernel_size=(3, 3), activation='relu'))
model1.add(MaxPooling2D((2, 2)))
model1.add(Conv2D(64, kernel_size=(3, 3), activation='relu'))
model1.add(MaxPooling2D((2, 2)))
# flattening the output of the conv layer after max pooling to make it ready for creating dense connections
model1.add(Flatten())
# Adding a fully connected dense layer with 16 neurons
model1.add(Dense(64, activation='relu'))
model1.add(Dropout(0.4))
# Adding the output layer with 12 neurons and activation functions as softmax since this is a multi-class classification problem
model1.add(Dense(12, activation='softmax'))
# Using SGD Optimizer
# opt = SGD(learning_rate=0.01, momentum=0.9)
opt=Adam()
# Compile model
model1.compile(optimizer=opt, loss='categorical_crossentropy', metrics=['accuracy'])
# Generating the summary of the model
model1.summary()
Model: "sequential"
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┓ ┃ Layer (type) ┃ Output Shape ┃ Param # ┃ ┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━┩ │ conv2d (Conv2D) │ (None, 126, 126, 32) │ 896 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ max_pooling2d (MaxPooling2D) │ (None, 63, 63, 32) │ 0 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ conv2d_1 (Conv2D) │ (None, 61, 61, 64) │ 18,496 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ max_pooling2d_1 (MaxPooling2D) │ (None, 30, 30, 64) │ 0 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ conv2d_2 (Conv2D) │ (None, 28, 28, 64) │ 36,928 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ max_pooling2d_2 (MaxPooling2D) │ (None, 14, 14, 64) │ 0 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ conv2d_3 (Conv2D) │ (None, 12, 12, 64) │ 36,928 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ max_pooling2d_3 (MaxPooling2D) │ (None, 6, 6, 64) │ 0 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ conv2d_4 (Conv2D) │ (None, 4, 4, 64) │ 36,928 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ max_pooling2d_4 (MaxPooling2D) │ (None, 2, 2, 64) │ 0 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ flatten (Flatten) │ (None, 256) │ 0 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ dense (Dense) │ (None, 64) │ 16,448 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ dropout (Dropout) │ (None, 64) │ 0 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ dense_1 (Dense) │ (None, 12) │ 780 │ └──────────────────────────────────────┴─────────────────────────────┴─────────────────┘
Total params: 147,404 (575.80 KB)
Trainable params: 147,404 (575.80 KB)
Non-trainable params: 0 (0.00 B)
callback_es = tf.keras.callbacks.EarlyStopping(monitor='val_accuracy', patience=20, min_delta=0.0001, restore_best_weights=True)
Fitting the model on train data¶
history = model1.fit(
train_datagen1.flow(X_train_normalized, y_train_encoded, seed=42, shuffle=True),
epochs=60,
validation_data=(X_val_normalized, y_val_encoded),
batch_size=32,
verbose=1,
callbacks=[callback_es])
Epoch 1/60 121/121 ━━━━━━━━━━━━━━━━━━━━ 13s 92ms/step - accuracy: 0.1673 - loss: 2.3368 - val_accuracy: 0.3224 - val_loss: 1.9645 Epoch 2/60 121/121 ━━━━━━━━━━━━━━━━━━━━ 11s 92ms/step - accuracy: 0.3390 - loss: 1.8291 - val_accuracy: 0.4042 - val_loss: 1.6134 Epoch 3/60 121/121 ━━━━━━━━━━━━━━━━━━━━ 11s 94ms/step - accuracy: 0.4112 - loss: 1.6411 - val_accuracy: 0.4439 - val_loss: 1.5044 Epoch 4/60 121/121 ━━━━━━━━━━━━━━━━━━━━ 11s 94ms/step - accuracy: 0.4714 - loss: 1.4943 - val_accuracy: 0.4930 - val_loss: 1.4253 Epoch 5/60 121/121 ━━━━━━━━━━━━━━━━━━━━ 12s 95ms/step - accuracy: 0.5055 - loss: 1.3512 - val_accuracy: 0.6262 - val_loss: 1.0779 Epoch 6/60 121/121 ━━━━━━━━━━━━━━━━━━━━ 11s 94ms/step - accuracy: 0.5750 - loss: 1.1891 - val_accuracy: 0.6612 - val_loss: 1.0131 Epoch 7/60 121/121 ━━━━━━━━━━━━━━━━━━━━ 11s 91ms/step - accuracy: 0.6146 - loss: 1.0839 - val_accuracy: 0.6285 - val_loss: 1.0328 Epoch 8/60 121/121 ━━━━━━━━━━━━━━━━━━━━ 11s 92ms/step - accuracy: 0.6435 - loss: 1.0078 - val_accuracy: 0.7126 - val_loss: 0.8234 Epoch 9/60 121/121 ━━━━━━━━━━━━━━━━━━━━ 11s 91ms/step - accuracy: 0.6607 - loss: 0.9300 - val_accuracy: 0.6939 - val_loss: 0.8777 Epoch 10/60 121/121 ━━━━━━━━━━━━━━━━━━━━ 11s 91ms/step - accuracy: 0.6730 - loss: 0.9037 - val_accuracy: 0.6706 - val_loss: 0.9063 Epoch 11/60 121/121 ━━━━━━━━━━━━━━━━━━━━ 11s 91ms/step - accuracy: 0.6966 - loss: 0.8713 - val_accuracy: 0.7523 - val_loss: 0.7496 Epoch 12/60 121/121 ━━━━━━━━━━━━━━━━━━━━ 11s 91ms/step - accuracy: 0.7128 - loss: 0.8162 - val_accuracy: 0.7664 - val_loss: 0.6761 Epoch 13/60 121/121 ━━━━━━━━━━━━━━━━━━━━ 11s 90ms/step - accuracy: 0.7374 - loss: 0.7429 - val_accuracy: 0.7827 - val_loss: 0.6893 Epoch 14/60 121/121 ━━━━━━━━━━━━━━━━━━━━ 11s 92ms/step - accuracy: 0.7486 - loss: 0.7266 - val_accuracy: 0.7921 - val_loss: 0.6222 Epoch 15/60 121/121 ━━━━━━━━━━━━━━━━━━━━ 11s 91ms/step - accuracy: 0.7676 - loss: 0.6264 - val_accuracy: 0.8318 - val_loss: 0.5579 Epoch 16/60 121/121 ━━━━━━━━━━━━━━━━━━━━ 11s 92ms/step - accuracy: 0.7933 - loss: 0.5990 - val_accuracy: 0.8201 - val_loss: 0.5970 Epoch 17/60 121/121 ━━━━━━━━━━━━━━━━━━━━ 11s 91ms/step - accuracy: 0.8117 - loss: 0.5618 - val_accuracy: 0.8248 - val_loss: 0.5239 Epoch 18/60 121/121 ━━━━━━━━━━━━━━━━━━━━ 11s 90ms/step - accuracy: 0.8129 - loss: 0.5424 - val_accuracy: 0.8341 - val_loss: 0.5148 Epoch 19/60 121/121 ━━━━━━━━━━━━━━━━━━━━ 11s 91ms/step - accuracy: 0.8047 - loss: 0.5400 - val_accuracy: 0.8224 - val_loss: 0.5354 Epoch 20/60 121/121 ━━━━━━━━━━━━━━━━━━━━ 11s 91ms/step - accuracy: 0.8124 - loss: 0.5347 - val_accuracy: 0.8458 - val_loss: 0.5002 Epoch 21/60 121/121 ━━━━━━━━━━━━━━━━━━━━ 11s 92ms/step - accuracy: 0.8260 - loss: 0.5116 - val_accuracy: 0.8341 - val_loss: 0.5318 Epoch 22/60 121/121 ━━━━━━━━━━━━━━━━━━━━ 11s 92ms/step - accuracy: 0.8255 - loss: 0.5059 - val_accuracy: 0.7991 - val_loss: 0.6077 Epoch 23/60 121/121 ━━━━━━━━━━━━━━━━━━━━ 11s 93ms/step - accuracy: 0.8203 - loss: 0.5013 - val_accuracy: 0.8294 - val_loss: 0.5315 Epoch 24/60 121/121 ━━━━━━━━━━━━━━━━━━━━ 11s 93ms/step - accuracy: 0.8296 - loss: 0.4822 - val_accuracy: 0.8762 - val_loss: 0.4368 Epoch 25/60 121/121 ━━━━━━━━━━━━━━━━━━━━ 11s 92ms/step - accuracy: 0.8457 - loss: 0.4307 - val_accuracy: 0.8598 - val_loss: 0.4601 Epoch 26/60 121/121 ━━━━━━━━━━━━━━━━━━━━ 11s 93ms/step - accuracy: 0.8547 - loss: 0.4229 - val_accuracy: 0.8692 - val_loss: 0.4334 Epoch 27/60 121/121 ━━━━━━━━━━━━━━━━━━━━ 11s 92ms/step - accuracy: 0.8419 - loss: 0.4208 - val_accuracy: 0.8715 - val_loss: 0.4182 Epoch 28/60 121/121 ━━━━━━━━━━━━━━━━━━━━ 11s 93ms/step - accuracy: 0.8581 - loss: 0.4021 - val_accuracy: 0.8598 - val_loss: 0.4848 Epoch 29/60 121/121 ━━━━━━━━━━━━━━━━━━━━ 11s 90ms/step - accuracy: 0.8547 - loss: 0.4118 - val_accuracy: 0.8248 - val_loss: 0.5490 Epoch 30/60 121/121 ━━━━━━━━━━━━━━━━━━━━ 11s 90ms/step - accuracy: 0.8572 - loss: 0.3857 - val_accuracy: 0.8855 - val_loss: 0.4121 Epoch 31/60 121/121 ━━━━━━━━━━━━━━━━━━━━ 11s 92ms/step - accuracy: 0.8570 - loss: 0.4008 - val_accuracy: 0.8808 - val_loss: 0.3892 Epoch 32/60 121/121 ━━━━━━━━━━━━━━━━━━━━ 11s 92ms/step - accuracy: 0.8643 - loss: 0.3591 - val_accuracy: 0.8621 - val_loss: 0.4287 Epoch 33/60 121/121 ━━━━━━━━━━━━━━━━━━━━ 11s 91ms/step - accuracy: 0.8674 - loss: 0.3665 - val_accuracy: 0.8738 - val_loss: 0.4338 Epoch 34/60 121/121 ━━━━━━━━━━━━━━━━━━━━ 11s 91ms/step - accuracy: 0.8594 - loss: 0.3749 - val_accuracy: 0.8692 - val_loss: 0.4538 Epoch 35/60 121/121 ━━━━━━━━━━━━━━━━━━━━ 11s 92ms/step - accuracy: 0.8846 - loss: 0.3269 - val_accuracy: 0.8808 - val_loss: 0.4001 Epoch 36/60 121/121 ━━━━━━━━━━━━━━━━━━━━ 11s 91ms/step - accuracy: 0.8781 - loss: 0.3417 - val_accuracy: 0.8738 - val_loss: 0.4083 Epoch 37/60 121/121 ━━━━━━━━━━━━━━━━━━━━ 11s 92ms/step - accuracy: 0.8659 - loss: 0.3473 - val_accuracy: 0.8738 - val_loss: 0.4490 Epoch 38/60 121/121 ━━━━━━━━━━━━━━━━━━━━ 11s 92ms/step - accuracy: 0.8758 - loss: 0.3322 - val_accuracy: 0.8785 - val_loss: 0.3907 Epoch 39/60 121/121 ━━━━━━━━━━━━━━━━━━━━ 11s 93ms/step - accuracy: 0.8749 - loss: 0.3317 - val_accuracy: 0.8832 - val_loss: 0.3886 Epoch 40/60 121/121 ━━━━━━━━━━━━━━━━━━━━ 11s 91ms/step - accuracy: 0.8762 - loss: 0.3302 - val_accuracy: 0.8738 - val_loss: 0.4124 Epoch 41/60 121/121 ━━━━━━━━━━━━━━━━━━━━ 11s 91ms/step - accuracy: 0.8775 - loss: 0.3302 - val_accuracy: 0.8855 - val_loss: 0.3810 Epoch 42/60 121/121 ━━━━━━━━━━━━━━━━━━━━ 11s 91ms/step - accuracy: 0.8828 - loss: 0.3158 - val_accuracy: 0.8855 - val_loss: 0.3606 Epoch 43/60 121/121 ━━━━━━━━━━━━━━━━━━━━ 11s 91ms/step - accuracy: 0.8781 - loss: 0.3289 - val_accuracy: 0.8785 - val_loss: 0.3780 Epoch 44/60 121/121 ━━━━━━━━━━━━━━━━━━━━ 11s 91ms/step - accuracy: 0.8869 - loss: 0.3027 - val_accuracy: 0.8692 - val_loss: 0.3717 Epoch 45/60 121/121 ━━━━━━━━━━━━━━━━━━━━ 11s 90ms/step - accuracy: 0.9007 - loss: 0.2668 - val_accuracy: 0.8925 - val_loss: 0.3632 Epoch 46/60 121/121 ━━━━━━━━━━━━━━━━━━━━ 11s 91ms/step - accuracy: 0.8876 - loss: 0.2920 - val_accuracy: 0.8575 - val_loss: 0.4948 Epoch 47/60 121/121 ━━━━━━━━━━━━━━━━━━━━ 11s 91ms/step - accuracy: 0.8729 - loss: 0.3530 - val_accuracy: 0.8785 - val_loss: 0.3838 Epoch 48/60 121/121 ━━━━━━━━━━━━━━━━━━━━ 11s 91ms/step - accuracy: 0.8909 - loss: 0.2890 - val_accuracy: 0.8598 - val_loss: 0.4077 Epoch 49/60 121/121 ━━━━━━━━━━━━━━━━━━━━ 11s 91ms/step - accuracy: 0.8799 - loss: 0.3279 - val_accuracy: 0.8692 - val_loss: 0.4255 Epoch 50/60 121/121 ━━━━━━━━━━━━━━━━━━━━ 11s 91ms/step - accuracy: 0.8905 - loss: 0.3029 - val_accuracy: 0.8879 - val_loss: 0.3923 Epoch 51/60 121/121 ━━━━━━━━━━━━━━━━━━━━ 11s 91ms/step - accuracy: 0.9045 - loss: 0.2473 - val_accuracy: 0.8925 - val_loss: 0.3635 Epoch 52/60 121/121 ━━━━━━━━━━━━━━━━━━━━ 11s 92ms/step - accuracy: 0.8908 - loss: 0.2875 - val_accuracy: 0.8785 - val_loss: 0.4018 Epoch 53/60 121/121 ━━━━━━━━━━━━━━━━━━━━ 11s 92ms/step - accuracy: 0.9037 - loss: 0.2575 - val_accuracy: 0.8879 - val_loss: 0.4274 Epoch 54/60 121/121 ━━━━━━━━━━━━━━━━━━━━ 11s 91ms/step - accuracy: 0.8757 - loss: 0.3358 - val_accuracy: 0.8785 - val_loss: 0.3921 Epoch 55/60 121/121 ━━━━━━━━━━━━━━━━━━━━ 11s 91ms/step - accuracy: 0.9090 - loss: 0.2475 - val_accuracy: 0.8902 - val_loss: 0.4336 Epoch 56/60 121/121 ━━━━━━━━━━━━━━━━━━━━ 12s 95ms/step - accuracy: 0.9076 - loss: 0.2406 - val_accuracy: 0.8808 - val_loss: 0.4163 Epoch 57/60 121/121 ━━━━━━━━━━━━━━━━━━━━ 11s 91ms/step - accuracy: 0.9009 - loss: 0.2716 - val_accuracy: 0.8949 - val_loss: 0.3282 Epoch 58/60 121/121 ━━━━━━━━━━━━━━━━━━━━ 11s 93ms/step - accuracy: 0.9062 - loss: 0.2480 - val_accuracy: 0.9019 - val_loss: 0.3606 Epoch 59/60 121/121 ━━━━━━━━━━━━━━━━━━━━ 11s 92ms/step - accuracy: 0.8990 - loss: 0.2681 - val_accuracy: 0.9112 - val_loss: 0.3189 Epoch 60/60 121/121 ━━━━━━━━━━━━━━━━━━━━ 11s 91ms/step - accuracy: 0.9048 - loss: 0.2387 - val_accuracy: 0.8995 - val_loss: 0.3399
Plot Accuracy Graph¶
plt.plot(history.history['accuracy'])
plt.plot(history.history['val_accuracy'])
plt.title('Model Accuracy')
plt.ylabel('Accuracy')
plt.xlabel('Epoch')
plt.legend(['Train', 'Validation'], loc='upper left')
plt.show()
- As we can see from above results, training and validation is giving better accuracy around 90%, let's evaluate the model on test data
Evaluate model on test data¶
test_accuracy1 = model1.evaluate(X_test_normalized, y_test_encoded, verbose=1)
15/15 ━━━━━━━━━━━━━━━━━━━━ 0s 18ms/step - accuracy: 0.9021 - loss: 0.3532
- As we can see model 1 is giving around 90% accuray on traning and validation set and also 90% test accuracy.
- Model is generalizing well
Generating predictions on test data¶
y_pred = model1.predict(X_test_normalized)
15/15 ━━━━━━━━━━━━━━━━━━━━ 0s 24ms/step
Plot Confusion matrix¶
# Obtaining the categorical values from y_test_encoded and y_pred
y_pred_arg=np.argmax(y_pred,axis=1)
y_test_arg=np.argmax(y_test_encoded,axis=1)
# Plotting the Confusion Matrix using confusion matrix() function which is also predefined tensorflow module
confusion_matrix = tf.math.confusion_matrix(y_test_arg,y_pred_arg)
f, ax = plt.subplots(figsize=(10, 8))
sns.heatmap(
confusion_matrix,
annot=True,
linewidths=.4,
fmt="d",
square=True,
ax=ax
)
# Setting the labels to both the axes
ax.set_xlabel('Predicted labels');ax.set_ylabel('True labels');
ax.set_title('Confusion Matrix');
ax.xaxis.set_ticklabels(list(lb.classes_),rotation=90)
ax.yaxis.set_ticklabels(list(lb.classes_),rotation=0)
plt.show()
- Considering confusion matrix, Model 1 is giving better predictions for Common Chickweed, Maize, Small-flowered Cranesbill,Scentless Mayweed, Fat hen
- Model has given misclassified predictions on Black grass, seems it confusing Black grass with Loose Silky-bent, also some wrong predictions for Loose Silky-bent as black grass
# Plotting Classification Report
from sklearn.metrics import classification_report
print(classification_report(y_test_arg, y_pred_arg, target_names=list(lb.classes_)))
precision recall f1-score support
Black-grass 0.64 0.35 0.45 26
Charlock 0.84 0.92 0.88 39
Cleavers 0.89 0.86 0.88 29
Common Chickweed 1.00 0.93 0.97 61
Common wheat 0.83 0.86 0.84 22
Fat Hen 0.90 0.96 0.93 48
Loose Silky-bent 0.80 0.94 0.87 65
Maize 0.92 1.00 0.96 22
Scentless Mayweed 0.91 0.96 0.93 52
Shepherds Purse 0.83 0.65 0.73 23
Small-flowered Cranesbill 0.96 0.98 0.97 50
Sugar beet 0.97 0.89 0.93 38
accuracy 0.89 475
macro avg 0.87 0.86 0.86 475
weighted avg 0.89 0.89 0.88 475
Comments¶
- Recall is low for Black-grass and except Shepherds Purse, for other classes recall is above 90%
- Precision is slighly low for Black grass but all other classes have good preicision and f1 score
- Even from confusion matrix, we see model did not perform well for black grass
- Overall accuracy is good, let's check if we can improve further
Model 2 - Model with Reducing Learning Rate & Data Augmentation¶
Reducing the Learning Rate:
Hint: Use ReduceLRonPlateau() function that will be used to decrease the learning rate by some factor, if the loss is not decreasing for some time. This may start decreasing the loss at a smaller learning rate. There is a possibility that the loss may still not decrease. This may lead to executing the learning rate reduction again in an attempt to achieve a lower loss.
learning_rate_reduction = ReduceLROnPlateau(monitor='val_accuracy', patience=3, verbose=1, factor=0.5, min_lr=0.00001)
# Clearing backend
from tensorflow.keras import backend
backend.clear_session()
# Fixing the seed for random number generators
import random
np.random.seed(42)
random.seed(42)
tf.random.set_seed(42)
# set the rotation_range to 20
train_datagen1 = ImageDataGenerator(rotation_range=20, # randomly rotate images in the range
horizontal_flip=True, # randomly flip images horizontally
vertical_flip=True # randomly flip images vertically
)
# Initializing a sequential model
# Initializing a sequential model
model2 = Sequential()
# Adding first conv layer with 128 filters and Kernel size 3*3, padding 'same'
model2.add(Conv2D(32, kernel_size=(3, 3), activation='relu', input_shape=(128,128,3)))
# Adding max pooling to reduce the size of output of first conv layer
model2.add(MaxPooling2D((2, 2)))
# Add two convolution and max-pooling layers activation = relu
model2.add(Conv2D(64, kernel_size=(3, 3), activation='relu'))
model2.add(layers.BatchNormalization())
model2.add(MaxPooling2D((2, 2)))
model2.add(Conv2D(64, kernel_size=(3, 3), activation='relu'))
model2.add(layers.BatchNormalization())
model2.add(MaxPooling2D((2, 2)))
model2.add(Conv2D(64, kernel_size=(3, 3), activation='relu'))
model2.add(layers.BatchNormalization())
model2.add(MaxPooling2D((2, 2)))
model2.add(Conv2D(64, kernel_size=(3, 3), activation='relu'))
model2.add(layers.BatchNormalization())
model2.add(MaxPooling2D((2, 2)))
# flattening the output of the conv layer after max pooling to make it ready for creating dense connections
model2.add(Flatten())
# Adding a fully connected dense layer with 16 neurons
model2.add(Dense(64, activation='relu'))
model2.add(Dropout(0.3))
# Adding the output layer with 12 neurons and activation functions as softmax since this is a multi-class classification problem
model2.add(Dense(12, activation='softmax'))
# Using SGD Optimizer
# opt = SGD(learning_rate=0.01, momentum=0.9)
opt=Adam()
# Compile model
model2.compile(optimizer=opt, loss='categorical_crossentropy', metrics=['accuracy'])
# Generating the summary of the model
model2.summary()
Model: "sequential"
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┓ ┃ Layer (type) ┃ Output Shape ┃ Param # ┃ ┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━┩ │ conv2d (Conv2D) │ (None, 126, 126, 32) │ 896 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ max_pooling2d (MaxPooling2D) │ (None, 63, 63, 32) │ 0 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ conv2d_1 (Conv2D) │ (None, 61, 61, 64) │ 18,496 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ batch_normalization │ (None, 61, 61, 64) │ 256 │ │ (BatchNormalization) │ │ │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ max_pooling2d_1 (MaxPooling2D) │ (None, 30, 30, 64) │ 0 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ conv2d_2 (Conv2D) │ (None, 28, 28, 64) │ 36,928 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ batch_normalization_1 │ (None, 28, 28, 64) │ 256 │ │ (BatchNormalization) │ │ │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ max_pooling2d_2 (MaxPooling2D) │ (None, 14, 14, 64) │ 0 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ conv2d_3 (Conv2D) │ (None, 12, 12, 64) │ 36,928 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ batch_normalization_2 │ (None, 12, 12, 64) │ 256 │ │ (BatchNormalization) │ │ │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ max_pooling2d_3 (MaxPooling2D) │ (None, 6, 6, 64) │ 0 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ conv2d_4 (Conv2D) │ (None, 4, 4, 64) │ 36,928 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ batch_normalization_3 │ (None, 4, 4, 64) │ 256 │ │ (BatchNormalization) │ │ │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ max_pooling2d_4 (MaxPooling2D) │ (None, 2, 2, 64) │ 0 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ flatten (Flatten) │ (None, 256) │ 0 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ dense (Dense) │ (None, 64) │ 16,448 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ dropout (Dropout) │ (None, 64) │ 0 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ dense_1 (Dense) │ (None, 12) │ 780 │ └──────────────────────────────────────┴─────────────────────────────┴─────────────────┘
Total params: 148,428 (579.80 KB)
Trainable params: 147,916 (577.80 KB)
Non-trainable params: 512 (2.00 KB)
Fitting the model¶
# fit the model on train data with batch_size=64 and epochs=30
epochs = 50
batch_size = 32
history = model2.fit(train_datagen.flow(X_train_normalized, y_train_encoded, seed=42, shuffle=True),
epochs=epochs,
batch_size=batch_size,
steps_per_epoch=X_train_normalized.shape[0] // batch_size,
validation_data=(X_val_normalized,y_val_encoded),
verbose=1,callbacks=[learning_rate_reduction])
Epoch 1/50 120/120 ━━━━━━━━━━━━━━━━━━━━ 16s 113ms/step - accuracy: 0.2576 - loss: 2.3839 - val_accuracy: 0.0467 - val_loss: 5.1001 - learning_rate: 0.0010 Epoch 2/50 120/120 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.4375 - loss: 1.6614 - val_accuracy: 0.0467 - val_loss: 5.1101 - learning_rate: 0.0010 Epoch 3/50 120/120 ━━━━━━━━━━━━━━━━━━━━ 14s 113ms/step - accuracy: 0.5388 - loss: 1.3733 - val_accuracy: 0.0935 - val_loss: 5.9076 - learning_rate: 0.0010 Epoch 4/50 120/120 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.7500 - loss: 0.8170 - val_accuracy: 0.0935 - val_loss: 5.9096 - learning_rate: 0.0010 Epoch 5/50 120/120 ━━━━━━━━━━━━━━━━━━━━ 14s 116ms/step - accuracy: 0.6419 - loss: 1.0307 - val_accuracy: 0.0771 - val_loss: 6.7895 - learning_rate: 0.0010 Epoch 6/50 1/120 ━━━━━━━━━━━━━━━━━━━━ 9s 83ms/step - accuracy: 0.6250 - loss: 1.1705 Epoch 6: ReduceLROnPlateau reducing learning rate to 0.0005000000237487257. 120/120 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.6250 - loss: 1.1705 - val_accuracy: 0.0771 - val_loss: 6.7641 - learning_rate: 0.0010 Epoch 7/50 120/120 ━━━━━━━━━━━━━━━━━━━━ 13s 112ms/step - accuracy: 0.7215 - loss: 0.8213 - val_accuracy: 0.0794 - val_loss: 6.8274 - learning_rate: 5.0000e-04 Epoch 8/50 120/120 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.5938 - loss: 0.8244 - val_accuracy: 0.0748 - val_loss: 6.9343 - learning_rate: 5.0000e-04 Epoch 9/50 120/120 ━━━━━━━━━━━━━━━━━━━━ 13s 111ms/step - accuracy: 0.7527 - loss: 0.6844 - val_accuracy: 0.1822 - val_loss: 3.1616 - learning_rate: 5.0000e-04 Epoch 10/50 120/120 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.8438 - loss: 0.4490 - val_accuracy: 0.1822 - val_loss: 3.2334 - learning_rate: 5.0000e-04 Epoch 11/50 120/120 ━━━━━━━━━━━━━━━━━━━━ 13s 110ms/step - accuracy: 0.7806 - loss: 0.6340 - val_accuracy: 0.7780 - val_loss: 0.6466 - learning_rate: 5.0000e-04 Epoch 12/50 120/120 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.8125 - loss: 0.4733 - val_accuracy: 0.7710 - val_loss: 0.6639 - learning_rate: 5.0000e-04 Epoch 13/50 120/120 ━━━━━━━━━━━━━━━━━━━━ 13s 110ms/step - accuracy: 0.8033 - loss: 0.5845 - val_accuracy: 0.7173 - val_loss: 0.8277 - learning_rate: 5.0000e-04 Epoch 14/50 1/120 ━━━━━━━━━━━━━━━━━━━━ 9s 83ms/step - accuracy: 0.7188 - loss: 0.6965 Epoch 14: ReduceLROnPlateau reducing learning rate to 0.0002500000118743628. 120/120 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.7188 - loss: 0.6965 - val_accuracy: 0.7383 - val_loss: 0.7942 - learning_rate: 5.0000e-04 Epoch 15/50 120/120 ━━━━━━━━━━━━━━━━━━━━ 13s 110ms/step - accuracy: 0.8192 - loss: 0.5330 - val_accuracy: 0.8458 - val_loss: 0.4674 - learning_rate: 2.5000e-04 Epoch 16/50 120/120 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.8438 - loss: 0.4077 - val_accuracy: 0.8458 - val_loss: 0.4868 - learning_rate: 2.5000e-04 Epoch 17/50 120/120 ━━━━━━━━━━━━━━━━━━━━ 13s 110ms/step - accuracy: 0.8275 - loss: 0.5009 - val_accuracy: 0.8388 - val_loss: 0.4613 - learning_rate: 2.5000e-04 Epoch 18/50 1/120 ━━━━━━━━━━━━━━━━━━━━ 10s 91ms/step - accuracy: 0.9062 - loss: 0.3323 Epoch 18: ReduceLROnPlateau reducing learning rate to 0.0001250000059371814. 120/120 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.9062 - loss: 0.3323 - val_accuracy: 0.8435 - val_loss: 0.4621 - learning_rate: 2.5000e-04 Epoch 19/50 120/120 ━━━━━━━━━━━━━━━━━━━━ 13s 110ms/step - accuracy: 0.8344 - loss: 0.4913 - val_accuracy: 0.8201 - val_loss: 0.4692 - learning_rate: 1.2500e-04 Epoch 20/50 120/120 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.7500 - loss: 0.7823 - val_accuracy: 0.8248 - val_loss: 0.4618 - learning_rate: 1.2500e-04 Epoch 21/50 120/120 ━━━━━━━━━━━━━━━━━━━━ 0s 107ms/step - accuracy: 0.8245 - loss: 0.4762 Epoch 21: ReduceLROnPlateau reducing learning rate to 6.25000029685907e-05. 120/120 ━━━━━━━━━━━━━━━━━━━━ 13s 110ms/step - accuracy: 0.8246 - loss: 0.4760 - val_accuracy: 0.8201 - val_loss: 0.4610 - learning_rate: 1.2500e-04 Epoch 22/50 120/120 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.7500 - loss: 0.5539 - val_accuracy: 0.8224 - val_loss: 0.4585 - learning_rate: 6.2500e-05 Epoch 23/50 120/120 ━━━━━━━━━━━━━━━━━━━━ 13s 109ms/step - accuracy: 0.8552 - loss: 0.4351 - val_accuracy: 0.8388 - val_loss: 0.4810 - learning_rate: 6.2500e-05 Epoch 24/50 1/120 ━━━━━━━━━━━━━━━━━━━━ 10s 86ms/step - accuracy: 0.8125 - loss: 0.4305 Epoch 24: ReduceLROnPlateau reducing learning rate to 3.125000148429535e-05. 120/120 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.8125 - loss: 0.4305 - val_accuracy: 0.8388 - val_loss: 0.4810 - learning_rate: 6.2500e-05 Epoch 25/50 120/120 ━━━━━━━━━━━━━━━━━━━━ 13s 109ms/step - accuracy: 0.8505 - loss: 0.4267 - val_accuracy: 0.8528 - val_loss: 0.4049 - learning_rate: 3.1250e-05 Epoch 26/50 120/120 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.8438 - loss: 0.5223 - val_accuracy: 0.8528 - val_loss: 0.4045 - learning_rate: 3.1250e-05 Epoch 27/50 120/120 ━━━━━━━━━━━━━━━━━━━━ 13s 110ms/step - accuracy: 0.8607 - loss: 0.4184 - val_accuracy: 0.8505 - val_loss: 0.4098 - learning_rate: 3.1250e-05 Epoch 28/50 1/120 ━━━━━━━━━━━━━━━━━━━━ 10s 85ms/step - accuracy: 0.8438 - loss: 0.4249 Epoch 28: ReduceLROnPlateau reducing learning rate to 1.5625000742147677e-05. 120/120 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.8438 - loss: 0.4249 - val_accuracy: 0.8505 - val_loss: 0.4103 - learning_rate: 3.1250e-05 Epoch 29/50 120/120 ━━━━━━━━━━━━━━━━━━━━ 13s 111ms/step - accuracy: 0.8486 - loss: 0.4248 - val_accuracy: 0.8575 - val_loss: 0.3996 - learning_rate: 1.5625e-05 Epoch 30/50 120/120 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.8125 - loss: 0.4711 - val_accuracy: 0.8575 - val_loss: 0.3993 - learning_rate: 1.5625e-05 Epoch 31/50 120/120 ━━━━━━━━━━━━━━━━━━━━ 13s 110ms/step - accuracy: 0.8475 - loss: 0.4188 - val_accuracy: 0.8668 - val_loss: 0.3914 - learning_rate: 1.5625e-05 Epoch 32/50 120/120 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.8125 - loss: 0.4689 - val_accuracy: 0.8668 - val_loss: 0.3911 - learning_rate: 1.5625e-05 Epoch 33/50 120/120 ━━━━━━━━━━━━━━━━━━━━ 13s 109ms/step - accuracy: 0.8483 - loss: 0.4209 - val_accuracy: 0.8621 - val_loss: 0.3941 - learning_rate: 1.5625e-05 Epoch 34/50 1/120 ━━━━━━━━━━━━━━━━━━━━ 10s 92ms/step - accuracy: 0.8438 - loss: 0.3642 Epoch 34: ReduceLROnPlateau reducing learning rate to 1e-05. 120/120 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.8438 - loss: 0.3642 - val_accuracy: 0.8621 - val_loss: 0.3935 - learning_rate: 1.5625e-05 Epoch 35/50 120/120 ━━━━━━━━━━━━━━━━━━━━ 13s 109ms/step - accuracy: 0.8495 - loss: 0.4235 - val_accuracy: 0.8621 - val_loss: 0.3854 - learning_rate: 1.0000e-05 Epoch 36/50 120/120 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.9375 - loss: 0.2938 - val_accuracy: 0.8621 - val_loss: 0.3859 - learning_rate: 1.0000e-05 Epoch 37/50 120/120 ━━━━━━━━━━━━━━━━━━━━ 13s 111ms/step - accuracy: 0.8459 - loss: 0.4025 - val_accuracy: 0.8621 - val_loss: 0.3887 - learning_rate: 1.0000e-05 Epoch 38/50 120/120 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.9688 - loss: 0.1799 - val_accuracy: 0.8598 - val_loss: 0.3887 - learning_rate: 1.0000e-05 Epoch 39/50 120/120 ━━━━━━━━━━━━━━━━━━━━ 13s 111ms/step - accuracy: 0.8495 - loss: 0.3958 - val_accuracy: 0.8551 - val_loss: 0.3954 - learning_rate: 1.0000e-05 Epoch 40/50 120/120 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.9375 - loss: 0.2187 - val_accuracy: 0.8551 - val_loss: 0.3956 - learning_rate: 1.0000e-05 Epoch 41/50 120/120 ━━━━━━━━━━━━━━━━━━━━ 13s 111ms/step - accuracy: 0.8516 - loss: 0.4143 - val_accuracy: 0.8621 - val_loss: 0.3881 - learning_rate: 1.0000e-05 Epoch 42/50 120/120 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.8438 - loss: 0.3724 - val_accuracy: 0.8621 - val_loss: 0.3881 - learning_rate: 1.0000e-05 Epoch 43/50 120/120 ━━━━━━━━━━━━━━━━━━━━ 13s 110ms/step - accuracy: 0.8503 - loss: 0.4286 - val_accuracy: 0.8621 - val_loss: 0.3913 - learning_rate: 1.0000e-05 Epoch 44/50 120/120 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.7188 - loss: 0.7387 - val_accuracy: 0.8621 - val_loss: 0.3917 - learning_rate: 1.0000e-05 Epoch 45/50 120/120 ━━━━━━━━━━━━━━━━━━━━ 14s 114ms/step - accuracy: 0.8566 - loss: 0.3949 - val_accuracy: 0.8575 - val_loss: 0.3864 - learning_rate: 1.0000e-05 Epoch 46/50 120/120 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.8750 - loss: 0.3060 - val_accuracy: 0.8575 - val_loss: 0.3869 - learning_rate: 1.0000e-05 Epoch 47/50 120/120 ━━━━━━━━━━━━━━━━━━━━ 13s 110ms/step - accuracy: 0.8632 - loss: 0.3852 - val_accuracy: 0.8598 - val_loss: 0.3980 - learning_rate: 1.0000e-05 Epoch 48/50 120/120 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.7812 - loss: 0.7183 - val_accuracy: 0.8598 - val_loss: 0.3982 - learning_rate: 1.0000e-05 Epoch 49/50 120/120 ━━━━━━━━━━━━━━━━━━━━ 13s 109ms/step - accuracy: 0.8502 - loss: 0.4178 - val_accuracy: 0.8598 - val_loss: 0.3914 - learning_rate: 1.0000e-05 Epoch 50/50 120/120 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.9688 - loss: 0.1456 - val_accuracy: 0.8598 - val_loss: 0.3912 - learning_rate: 1.0000e-05
Plot Accuracy Graph¶
plt.plot(history.history['accuracy'])
plt.plot(history.history['val_accuracy'])
plt.title('Model Accuracy')
plt.ylabel('Accuracy')
plt.xlabel('Epoch')
plt.legend(['Train', 'Validation'], loc='upper left')
plt.show()
- As we can see the accuracy has been improved with data augmentation compare to earlier model
Evaluate the model on test data¶
accuracy = model2.evaluate(X_test_normalized, y_test_encoded, verbose=2)
15/15 - 0s - 22ms/step - accuracy: 0.8505 - loss: 0.4334
Generating the predictions for test data¶
# Here we would get the output as probablities for each category
y_pred=model2.predict(X_test_normalized)
15/15 ━━━━━━━━━━━━━━━━━━━━ 1s 29ms/step
# Obtaining the categorical values from y_test_encoded and y_pred
y_pred_arg=np.argmax(y_pred,axis=1)
y_test_arg=np.argmax(y_test_encoded,axis=1)
# Plotting the Confusion Matrix using confusion matrix() function which is also predefined tensorflow module
confusion_matrix = tf.math.confusion_matrix(y_test_arg,y_pred_arg)
f, ax = plt.subplots(figsize=(10, 8))
sns.heatmap(
confusion_matrix,
annot=True,
linewidths=.4,
fmt="d",
square=True,
ax=ax
)
# Setting the labels to both the axes
ax.set_xlabel('Predicted labels');ax.set_ylabel('True labels');
ax.set_title('Confusion Matrix');
ax.xaxis.set_ticklabels(list(lb.classes_),rotation=90)
ax.yaxis.set_ticklabels(list(lb.classes_),rotation=0)
plt.show()
# Plotting Classification Report
from sklearn.metrics import classification_report
print(classification_report(y_test_arg, y_pred_arg, target_names=list(lb.classes_)))
precision recall f1-score support
Black-grass 0.33 0.19 0.24 26
Charlock 0.86 0.95 0.90 39
Cleavers 0.89 0.83 0.86 29
Common Chickweed 0.93 0.93 0.93 61
Common wheat 0.79 0.50 0.61 22
Fat Hen 0.94 0.92 0.93 48
Loose Silky-bent 0.71 0.92 0.81 65
Maize 0.81 1.00 0.90 22
Scentless Mayweed 0.85 0.96 0.90 52
Shepherds Purse 1.00 0.57 0.72 23
Small-flowered Cranesbill 0.94 0.92 0.93 50
Sugar beet 0.97 0.92 0.95 38
accuracy 0.85 475
macro avg 0.84 0.80 0.81 475
weighted avg 0.85 0.85 0.84 475
- The test accuracy of the model is around 85% which is less than earlier model
- Recall of brack grass is also reduced so
- Overall model performance is less than earlier model
Model 3 - Using VGG16 architecture¶
from tensorflow.keras.models import Model
from keras.applications.vgg16 import VGG16
vgg_model = VGG16(weights='imagenet', include_top = False, input_shape = (128,128,3))
vgg_model.summary()
Model: "vgg16"
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┓ ┃ Layer (type) ┃ Output Shape ┃ Param # ┃ ┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━┩ │ input_layer_3 (InputLayer) │ (None, 128, 128, 3) │ 0 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ block1_conv1 (Conv2D) │ (None, 128, 128, 64) │ 1,792 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ block1_conv2 (Conv2D) │ (None, 128, 128, 64) │ 36,928 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ block1_pool (MaxPooling2D) │ (None, 64, 64, 64) │ 0 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ block2_conv1 (Conv2D) │ (None, 64, 64, 128) │ 73,856 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ block2_conv2 (Conv2D) │ (None, 64, 64, 128) │ 147,584 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ block2_pool (MaxPooling2D) │ (None, 32, 32, 128) │ 0 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ block3_conv1 (Conv2D) │ (None, 32, 32, 256) │ 295,168 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ block3_conv2 (Conv2D) │ (None, 32, 32, 256) │ 590,080 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ block3_conv3 (Conv2D) │ (None, 32, 32, 256) │ 590,080 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ block3_pool (MaxPooling2D) │ (None, 16, 16, 256) │ 0 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ block4_conv1 (Conv2D) │ (None, 16, 16, 512) │ 1,180,160 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ block4_conv2 (Conv2D) │ (None, 16, 16, 512) │ 2,359,808 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ block4_conv3 (Conv2D) │ (None, 16, 16, 512) │ 2,359,808 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ block4_pool (MaxPooling2D) │ (None, 8, 8, 512) │ 0 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ block5_conv1 (Conv2D) │ (None, 8, 8, 512) │ 2,359,808 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ block5_conv2 (Conv2D) │ (None, 8, 8, 512) │ 2,359,808 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ block5_conv3 (Conv2D) │ (None, 8, 8, 512) │ 2,359,808 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ block5_pool (MaxPooling2D) │ (None, 4, 4, 512) │ 0 │ └──────────────────────────────────────┴─────────────────────────────┴─────────────────┘
Total params: 14,714,688 (56.13 MB)
Trainable params: 14,714,688 (56.13 MB)
Non-trainable params: 0 (0.00 B)
# Making all the layers of the VGG model non-trainable. i.e. freezing them
for layer in vgg_model.layers:
layer.trainable = False
new_model = Sequential()
# Adding the convolutional part of the VGG16 model from above
new_model.add(vgg_model)
# Flattening the output of the VGG16 model because it is from a convolutional layer
new_model.add(Flatten())
# Adding a dense output layer
new_model.add(layers.BatchNormalization())
new_model.add(Dense(32, activation='relu'))
new_model.add(Dropout(0.2))
new_model.add(layers.BatchNormalization())
new_model.add(Dense(16, activation='relu'))
new_model.add(Dense(12, activation='softmax'))
opt=Adam(learning_rate=0.0001, beta_1=0.9, beta_2=0.999)
# Compile model
new_model.compile(optimizer=opt, loss='categorical_crossentropy', metrics=['accuracy'])
# Generating the summary of the model
new_model.summary()
Model: "sequential_2"
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┓ ┃ Layer (type) ┃ Output Shape ┃ Param # ┃ ┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━┩ │ vgg16 (Functional) │ (None, 4, 4, 512) │ 14,714,688 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ flatten_2 (Flatten) │ (None, 8192) │ 0 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ batch_normalization_6 │ (None, 8192) │ 32,768 │ │ (BatchNormalization) │ │ │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ dense_5 (Dense) │ (None, 32) │ 262,176 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ dropout_2 (Dropout) │ (None, 32) │ 0 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ batch_normalization_7 │ (None, 32) │ 128 │ │ (BatchNormalization) │ │ │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ dense_6 (Dense) │ (None, 16) │ 528 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ dense_7 (Dense) │ (None, 12) │ 204 │ └──────────────────────────────────────┴─────────────────────────────┴─────────────────┘
Total params: 15,010,492 (57.26 MB)
Trainable params: 279,356 (1.07 MB)
Non-trainable params: 14,731,136 (56.19 MB)
callback_es = tf.keras.callbacks.EarlyStopping(monitor='val_accuracy', patience=10, min_delta=0.0001, restore_best_weights=True)
# Epochs
epochs = 50
# Batch size
batch_size = 32
history_vgg16 = new_model.fit(train_datagen1.flow(X_train_normalized, y_train_encoded,
seed=42,
shuffle=True),
epochs=epochs,
batch_size=batch_size,
steps_per_epoch=X_train_normalized.shape[0] // batch_size,
validation_data=(X_val_normalized,y_val_encoded),
verbose=1,callbacks=[callback_es])
Epoch 1/50 120/120 ━━━━━━━━━━━━━━━━━━━━ 69s 563ms/step - accuracy: 0.2206 - loss: 2.2362 - val_accuracy: 0.4813 - val_loss: 1.9825 Epoch 2/50 120/120 ━━━━━━━━━━━━━━━━━━━━ 8s 59ms/step - accuracy: 0.4375 - loss: 1.6743 - val_accuracy: 0.4836 - val_loss: 1.9779 Epoch 3/50 120/120 ━━━━━━━━━━━━━━━━━━━━ 69s 577ms/step - accuracy: 0.4755 - loss: 1.6552 - val_accuracy: 0.6332 - val_loss: 1.5963 Epoch 4/50 120/120 ━━━━━━━━━━━━━━━━━━━━ 7s 58ms/step - accuracy: 0.5938 - loss: 1.3362 - val_accuracy: 0.6355 - val_loss: 1.5940 Epoch 5/50 120/120 ━━━━━━━━━━━━━━━━━━━━ 70s 582ms/step - accuracy: 0.5722 - loss: 1.4546 - val_accuracy: 0.6682 - val_loss: 1.3327 Epoch 6/50 120/120 ━━━━━━━━━━━━━━━━━━━━ 8s 60ms/step - accuracy: 0.5312 - loss: 1.4326 - val_accuracy: 0.6706 - val_loss: 1.3305 Epoch 7/50 120/120 ━━━━━━━━━━━━━━━━━━━━ 70s 583ms/step - accuracy: 0.6226 - loss: 1.3176 - val_accuracy: 0.7033 - val_loss: 1.1366 Epoch 8/50 120/120 ━━━━━━━━━━━━━━━━━━━━ 8s 60ms/step - accuracy: 0.8438 - loss: 1.0012 - val_accuracy: 0.6986 - val_loss: 1.1357 Epoch 9/50 120/120 ━━━━━━━━━━━━━━━━━━━━ 70s 585ms/step - accuracy: 0.6634 - loss: 1.2184 - val_accuracy: 0.7313 - val_loss: 1.0107 Epoch 10/50 120/120 ━━━━━━━━━━━━━━━━━━━━ 8s 59ms/step - accuracy: 0.7188 - loss: 1.0065 - val_accuracy: 0.7313 - val_loss: 1.0094 Epoch 11/50 120/120 ━━━━━━━━━━━━━━━━━━━━ 70s 582ms/step - accuracy: 0.6904 - loss: 1.1052 - val_accuracy: 0.7523 - val_loss: 0.9257 Epoch 12/50 120/120 ━━━━━━━━━━━━━━━━━━━━ 8s 59ms/step - accuracy: 0.7188 - loss: 1.0463 - val_accuracy: 0.7523 - val_loss: 0.9249 Epoch 13/50 120/120 ━━━━━━━━━━━━━━━━━━━━ 70s 582ms/step - accuracy: 0.7013 - loss: 1.0559 - val_accuracy: 0.7710 - val_loss: 0.8456 Epoch 14/50 120/120 ━━━━━━━━━━━━━━━━━━━━ 8s 59ms/step - accuracy: 0.6250 - loss: 1.1622 - val_accuracy: 0.7710 - val_loss: 0.8449 Epoch 15/50 120/120 ━━━━━━━━━━━━━━━━━━━━ 70s 583ms/step - accuracy: 0.7384 - loss: 0.9733 - val_accuracy: 0.7757 - val_loss: 0.8102 Epoch 16/50 120/120 ━━━━━━━━━━━━━━━━━━━━ 8s 59ms/step - accuracy: 0.6562 - loss: 0.9414 - val_accuracy: 0.7757 - val_loss: 0.8095 Epoch 17/50 120/120 ━━━━━━━━━━━━━━━━━━━━ 70s 584ms/step - accuracy: 0.7520 - loss: 0.9019 - val_accuracy: 0.7967 - val_loss: 0.7381 Epoch 18/50 120/120 ━━━━━━━━━━━━━━━━━━━━ 8s 59ms/step - accuracy: 0.8125 - loss: 0.7886 - val_accuracy: 0.7967 - val_loss: 0.7377 Epoch 19/50 120/120 ━━━━━━━━━━━━━━━━━━━━ 70s 583ms/step - accuracy: 0.7526 - loss: 0.8858 - val_accuracy: 0.8061 - val_loss: 0.6941 Epoch 20/50 120/120 ━━━━━━━━━━━━━━━━━━━━ 8s 59ms/step - accuracy: 0.7500 - loss: 0.9106 - val_accuracy: 0.8061 - val_loss: 0.6937 Epoch 21/50 120/120 ━━━━━━━━━━━━━━━━━━━━ 70s 584ms/step - accuracy: 0.7927 - loss: 0.7812 - val_accuracy: 0.8201 - val_loss: 0.6541 Epoch 22/50 120/120 ━━━━━━━━━━━━━━━━━━━━ 8s 60ms/step - accuracy: 0.7812 - loss: 0.7311 - val_accuracy: 0.8201 - val_loss: 0.6537 Epoch 23/50 120/120 ━━━━━━━━━━━━━━━━━━━━ 70s 584ms/step - accuracy: 0.7908 - loss: 0.7521 - val_accuracy: 0.8248 - val_loss: 0.6254 Epoch 24/50 120/120 ━━━━━━━━━━━━━━━━━━━━ 8s 60ms/step - accuracy: 0.7500 - loss: 0.7910 - val_accuracy: 0.8271 - val_loss: 0.6250 Epoch 25/50 120/120 ━━━━━━━━━━━━━━━━━━━━ 70s 584ms/step - accuracy: 0.7975 - loss: 0.7030 - val_accuracy: 0.8224 - val_loss: 0.6059 Epoch 26/50 120/120 ━━━━━━━━━━━━━━━━━━━━ 7s 59ms/step - accuracy: 0.6875 - loss: 0.9090 - val_accuracy: 0.8248 - val_loss: 0.6057 Epoch 27/50 120/120 ━━━━━━━━━━━━━━━━━━━━ 70s 582ms/step - accuracy: 0.7976 - loss: 0.7051 - val_accuracy: 0.8388 - val_loss: 0.5781 Epoch 28/50 120/120 ━━━━━━━━━━━━━━━━━━━━ 8s 59ms/step - accuracy: 0.9375 - loss: 0.3988 - val_accuracy: 0.8364 - val_loss: 0.5782 Epoch 29/50 120/120 ━━━━━━━━━━━━━━━━━━━━ 70s 583ms/step - accuracy: 0.8076 - loss: 0.6738 - val_accuracy: 0.8435 - val_loss: 0.5445 Epoch 30/50 120/120 ━━━━━━━━━━━━━━━━━━━━ 8s 60ms/step - accuracy: 0.9688 - loss: 0.3688 - val_accuracy: 0.8458 - val_loss: 0.5448 Epoch 31/50 120/120 ━━━━━━━━━━━━━━━━━━━━ 70s 587ms/step - accuracy: 0.8258 - loss: 0.6305 - val_accuracy: 0.8481 - val_loss: 0.5390 Epoch 32/50 120/120 ━━━━━━━━━━━━━━━━━━━━ 8s 60ms/step - accuracy: 0.6562 - loss: 0.8649 - val_accuracy: 0.8505 - val_loss: 0.5381 Epoch 33/50 120/120 ━━━━━━━━━━━━━━━━━━━━ 70s 586ms/step - accuracy: 0.8321 - loss: 0.5788 - val_accuracy: 0.8411 - val_loss: 0.5211 Epoch 34/50 120/120 ━━━━━━━━━━━━━━━━━━━━ 8s 60ms/step - accuracy: 0.8438 - loss: 0.4275 - val_accuracy: 0.8411 - val_loss: 0.5206 Epoch 35/50 120/120 ━━━━━━━━━━━━━━━━━━━━ 70s 588ms/step - accuracy: 0.8394 - loss: 0.5793 - val_accuracy: 0.8481 - val_loss: 0.5161 Epoch 36/50 120/120 ━━━━━━━━━━━━━━━━━━━━ 8s 59ms/step - accuracy: 0.7500 - loss: 0.6967 - val_accuracy: 0.8481 - val_loss: 0.5159 Epoch 37/50 120/120 ━━━━━━━━━━━━━━━━━━━━ 70s 586ms/step - accuracy: 0.8258 - loss: 0.5699 - val_accuracy: 0.8505 - val_loss: 0.4921 Epoch 38/50 120/120 ━━━━━━━━━━━━━━━━━━━━ 8s 59ms/step - accuracy: 0.7812 - loss: 0.5682 - val_accuracy: 0.8505 - val_loss: 0.4919 Epoch 39/50 120/120 ━━━━━━━━━━━━━━━━━━━━ 70s 587ms/step - accuracy: 0.8394 - loss: 0.5317 - val_accuracy: 0.8598 - val_loss: 0.4757 Epoch 40/50 120/120 ━━━━━━━━━━━━━━━━━━━━ 8s 59ms/step - accuracy: 0.8438 - loss: 0.5973 - val_accuracy: 0.8598 - val_loss: 0.4750 Epoch 41/50 120/120 ━━━━━━━━━━━━━━━━━━━━ 70s 583ms/step - accuracy: 0.8367 - loss: 0.5417 - val_accuracy: 0.8575 - val_loss: 0.4593 Epoch 42/50 120/120 ━━━━━━━━━━━━━━━━━━━━ 8s 60ms/step - accuracy: 0.9062 - loss: 0.4695 - val_accuracy: 0.8551 - val_loss: 0.4585 Epoch 43/50 120/120 ━━━━━━━━━━━━━━━━━━━━ 70s 585ms/step - accuracy: 0.8696 - loss: 0.4777 - val_accuracy: 0.8598 - val_loss: 0.4504 Epoch 44/50 120/120 ━━━━━━━━━━━━━━━━━━━━ 8s 60ms/step - accuracy: 0.8125 - loss: 0.4858 - val_accuracy: 0.8598 - val_loss: 0.4505 Epoch 45/50 120/120 ━━━━━━━━━━━━━━━━━━━━ 72s 598ms/step - accuracy: 0.8548 - loss: 0.4925 - val_accuracy: 0.8575 - val_loss: 0.4366 Epoch 46/50 120/120 ━━━━━━━━━━━━━━━━━━━━ 8s 59ms/step - accuracy: 0.8125 - loss: 0.5229 - val_accuracy: 0.8528 - val_loss: 0.4363 Epoch 47/50 120/120 ━━━━━━━━━━━━━━━━━━━━ 70s 582ms/step - accuracy: 0.8653 - loss: 0.4448 - val_accuracy: 0.8458 - val_loss: 0.4315 Epoch 48/50 120/120 ━━━━━━━━━━━━━━━━━━━━ 8s 59ms/step - accuracy: 0.9688 - loss: 0.3226 - val_accuracy: 0.8435 - val_loss: 0.4326 Epoch 49/50 120/120 ━━━━━━━━━━━━━━━━━━━━ 70s 583ms/step - accuracy: 0.8486 - loss: 0.4846 - val_accuracy: 0.8505 - val_loss: 0.4192
plt.plot(history_vgg16.history['accuracy'])
plt.plot(history_vgg16.history['val_accuracy'])
plt.title('Model Accuracy')
plt.ylabel('Accuracy')
plt.xlabel('Epoch')
plt.legend(['Train', 'Validation'], loc='upper left')
plt.show()
accuracy = new_model.evaluate(X_test_normalized, y_test_encoded, verbose=1)
15/15 ━━━━━━━━━━━━━━━━━━━━ 7s 459ms/step - accuracy: 0.8599 - loss: 0.4553
As VGG16 models does not perform better than earlier model, let's skip the furthern evalution of model on test data
Final Model¶
pd.DataFrame({'Models':['Model 0 - Base Model with CNN layers','Model 1 - Data Augmentation and Callback ','Model 2 -Data Augmentation and Reducing learning rate', 'Model 3 - Transfer learning with VGG16 architecture'],'Train Accuracy':['89%','91%','96%','85%'],'Validation Accuracy':['89%','90%','86%','85%'],'Test Accuracy':['88%','90%','85%','86%']})
| Models | Train Accuracy | Validation Accuracy | Test Accuracy | |
|---|---|---|---|---|
| 0 | Model 0 - Base Model with CNN layers | 89% | 89% | 88% |
| 1 | Model 1 - Data Augmentation and Callback | 91% | 90% | 90% |
| 2 | Model 2 -Data Augmentation and Reducing learni... | 96% | 86% | 85% |
| 3 | Model 3 - Transfer learning with VGG16 archite... | 85% | 85% | 86% |
- Considering the accuracy of all above models. 'Model 1 - Data Augementation and Callback' gives better accuracy score for train, validation and test sets i.e. around 90% accuracy, so will go ahead with Model 1 as best model.
- Let's visualize and predict the test data samples using Model 1
Visualizing the prediction¶
# Visualizing the predicted and correct label of images from test data
plt.figure(figsize=(2,2))
plt.imshow(X_test[2])
plt.show()
print('Predicted Label', lb.inverse_transform(model1.predict((X_test_normalized[2].reshape(1,128,128,3))))) # reshaping the input image as we are only trying to predict using a single image
print('True Label', lb.inverse_transform(y_test_encoded)[2]) # using inverse_transform() to get the output label from the output vector
plt.figure(figsize=(2,2))
plt.imshow(X_test[33])
plt.show()
print('Predicted Label', lb.inverse_transform(model1.predict((X_test_normalized[33].reshape(1,128,128,3))))) # reshaping the input image as we are only trying to predict using a single image
print('True Label', lb.inverse_transform(y_test_encoded)[33]) # using inverse_transform() to get the output label from the output vector
plt.figure(figsize=(2,2))
plt.imshow(X_test[36])
plt.show()
print('Predicted Label', lb.inverse_transform(model1.predict((X_test_normalized[36].reshape(1,128,128,3))))) # reshaping the input image as we are only trying to predict using a single image
print('True Label', lb.inverse_transform(y_test_encoded)[36])
1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 24ms/step Predicted Label ['Small-flowered Cranesbill'] True Label Small-flowered Cranesbill
1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 51ms/step Predicted Label ['Cleavers'] True Label Cleavers
1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 28ms/step Predicted Label ['Shepherds Purse'] True Label Shepherds Purse
Conclusion:¶
As we can see from predictions model has predicted all 3 random classes correctly. With some changes the parameters model may perform better.
We have built a CNN-model to predict the class of a plant, which works quite well. (Increasing number of epochs and/or adding layers to a model can even increase the performance)
CNN with Batch Normalization, Maxpooling, dropouts + Dense layers is a good combination for image classification
Actionable Insights and Business Recommendations¶
Actionable Insights¶
- Analyze classification accuracy and confusion matrix to identify misclassified seedling classes.
- Models can be further improved by training on different parameters and find better optimized combination of parameters
- Assess dataset balance and image quality to address potential biases and improve model training.
- Evaluate prediction speed for suitability in real-time applications.
- Incorporate active learning and retraining with new data to enhance model robustness.
- With better computational resources models can be further trained and improved on better image quality or larger sizes.
- Different transfer learning achitectures can be used and train to check for better results
- Model can be further improved by different sets of data augementation sets/combinations
Business Recommendations¶
- Companies can gather more data for various variety of seedlings to improve model performance.
- Precision Agriculture: Use the model for automated weed detection, crop monitoring, and early disease identification.
- Cost Optimization: Automate labor-intensive tasks like seedling identification to save costs and resources.
- Product Development: Create a mobile app for instant seedling classification and care advice.
- Revenue Streams: Offer subscription services for advanced features and license anonymized data for research.
- Partnerships: Collaborate with agricultural bodies and seed companies to tailor and scale the solution.
- Custom Solutions for Seed Companies:Partner with seed companies to customize the CNN model for their proprietary seed varieties, offering them a competitive edge.